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MEMORANDUM  FOR  DISTRIBUTION 

SUBJECT :  Key  Strategies  and  Practices  for  Oversight  of  DoD  Contracted  Audit  Services 
(Report  No.  D-2009-6-003) 

We  are  providing  this  report  for  your  information  and  use. 

We  performed  this  review  in  part  to  examine  how  DoD  Components  manage  and 
conduct  oversight  of  contracted  audit  services  to  ensure  contractors  meet  contract 
requirements  and  follow  Government  Auditing  Standards.  To  address  this,  we 
determined  the  extent  of  DoD’s  use  of  contractors  to  perform  audit  services.  From  this 
information,  we  selected  five  contracts  for  which  we  performed  a  detailed  review  of  the 
capabilities  and  practices  in  overseeing  the  contractors.  This  report  summarizes  what  we 
observed  and  identifies  certain  key  strategies  and  practices  we  believe  are  essential  to 
effective  and  efficient  oversight  of  contracted  audit  services. 

We  appreciate  the  courtesies  extended  to  the  staff.  Comments  or  questions  should 
be  directed  to  Mr.  Robert  L.  Kienitz  at  (703)  604-8754  (DSN  664-8754), 
robert.kienitz@dodig.mil. 


Carolyn  R.  Davis 
Assistant  Inspector  General 
for  Audit  Policy  and  Oversight 
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Foreword 


DoD  Instruction  7600.02,  “Audit  Policies,”  permits  DoD  Components  to  contract  for  audit 
services  when  applicable  expertise  is  unavailable  within  the  DoD  audit  organization, 
augmentation  of  the  DoD  audit  organization’s  audit  staff  is  necessary  to  execute  the  annual  audit 
plan,  or  temporary  audit  assistance  is  required  to  meet  audit  reporting  requirements  mandated  by 
law  or  DoD  Regulation.  The  Inspector  General  Act  of  1978,  as  amended,  tasks  the  Office  of  the 
Inspector  General  of  the  Department  of  Defense  with  taking  appropriate  steps  to  assure  that  any 
work  performed  by  non-Federal  auditors  complies  with  the  Government  Auditing  Standards 
established  by  the  Comptroller  General  of  the  United  States  for  audits  of  Federal  establishments, 
organizations,  programs,  activities,  and  functions. 

This  report  summarizes  what  we  observed  from  the  detailed  review  of  the  capabilities  and 
practices  in  overseeing  five  DoD  contracts  for  audit  services,  as  well  as  suggests  certain  key 
strategies  and  practices  we  believe  are  essential  to  effective  and  efficient  oversight.  It  discusses 

(1)  the  attributes  of  a  qualified  contracting  officer’s  representative,  (2)  oversight  planning, 

(3)  audit  monitoring,  (4)  monitoring  contractor  independence,  and  (5)  monitoring  contractor 
qualifications.  It  also  highlights  the  results  of  our  discussions  with  other  Federal  offices  of 
inspectors  general  about  the  practices  they  employ  when  overseeing  audit  services  contracts  that 
could  yield  benefits  to  the  DoD.  Highlights  cover  (1)  communicating  expectations  and  results, 

(2)  establishing  detailed  milestones,  (3)  tracking  deliverables,  and  (4)  determining  lessons 
learned. 

This  report  is  intended  for  use  by  DoD  contracting  and  contract  oversight  officials.  While  the 
suggestions  presented  are  not  mandatory,  we  hope  that  employing  them  will  guide  the  DoD 
officials  towards  taking  appropriate  steps  to  monitor  and  evaluate  contractor  perfonnance  early 
on  and  as  the  audit  progresses,  thereby  addressing  and  resolving  problems  that  may  result  in 
reduced  audit  quality,  missed  deadlines,  or  additional  costs  before  the  audit  is  completed.  Doing 
so  will  help  ensure  that  the  DoD  entities  contracting  for  or  requesting  audit  services  receive 
quality,  timely  audit  results  that  can  be  relied  upon  for  decision  making  purposes. 
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Introduction 


Our  review  objectives  were  to  determine  the  effectiveness  of  and  lessons  learned  from 
the  oversight  of  DoD  contracted  audit  services  and  to  identify  other  Federal  agencies’ 
practices  that  could  yield  benefits  to  the  DoD.  We  determined  the  extent  of  DoD’s  use  of 
audit  contractors  by  analyzing  procurement  data  on  contract  actions  for  auditing  services. 
We  also  identified  audit  services  contracts  for  which  the  DoD  Office  of  the  Inspector 
General  is  overseeing.  For  selected  contracts,  we  performed  a  detailed  review  of  the 
capabilities  and  practices  to  oversee  them.  This  report  summarizes  the  results  of  our 
review.  See  Appendix  A  for  a  discussion  of  the  scope  and  methodology  and  for  prior 
coverage  related  to  the  objectives.  This  report  also  highlights  the  results  of  our 
discussions  with  other  Federal  offices  of  inspectors  general  about  their  use  of  contractors 
to  perfonn  audit  services  and  their  practices  and  experiences  overseeing  audit  services 
contracts. 

Background 

Contract  Oversight.  Audit  work  and  reporting  quality,  timeliness,  and  compliance  with 
applicable  auditing  standards  are  essential  elements  of  successful  performance  under  an 
audit  services  contract.  Careful  oversight  of  contractor  performance  can  increase  the 
likelihood  that  these  goals  are  met.  This  means  keeping  well-informed  of  what  the 
contractor  is  doing;  using  technical  expertise  to  identify  contractor  actions  or  failures  to 
act  that  clearly  affect  the  quality,  progress,  or  cost  of  the  work;  calling  the  contractor’s 
attention  to  deficiencies;  and  detennining  appropriate  actions  to  deal  with  deficiencies. 
These  functions  are  typically  delegated  to  a  contracting  officer’s  representative  (COR), 
and  the  level  of  effort  that  will  be  necessary  and  the  techniques  that  will  be  used  should 
be  addressed  in  a  COR  oversight  plan.  The  type  of  effort  that  is  appropriate  depends  on 
the  complexity  and  scope  of  the  contract,  as  well  as  the  contract’s  specific  requirements 
for  monitoring,  inspection,  and  acceptance.  Factors  influencing  the  degree  of  oversight 
include  the  type  of  contract,  criticality  of  the  requirements,  contract  performance 
schedule,  contractor’s  experience  with  providing  the  services,  contractor’s  performance 
history,  and  the  level  of  the  contractor’s  own  inspection  system. 

Inspector  General  Oversight.  An  Office  of  Inspector  General  (OIG)  may  (1)  contract 
with  an  independent  public  accountant  (IP A)  to  perfonn  parts  of  or  an  entire  audit  or 
(2)  use  the  work  of  an  IP  A  contracted  by  another  entity.  For  financial  audits,  the  Office 
of  Management  and  Budget1  encourages  OIGs  to  use  section  650,  “Using  the  Work  of 
Others,”  of  the  Government  Accountability  Office/President’s  Council  on  Integrity  and 
Efficiency  (GAO/PCIE)  Financial  Audit  Manual  (FAM)  to  help  them  design  and  perfonn 
oversight  procedures  when  using  the  work  of  IP  As.  FAM  650  provides  that,  during 


1  Office  of  Management  and  Budget  (OMB)  Bulletin  Nos.  06-03  and  07-04,  “Audit  Requirements  for  Federal 
Financial  Statements.” 
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planning,  OIG  auditors  should  decide  on  the  amount  of  assurance  or  degree  of 
responsibility  they  wish  to  provide  or  accept  for  the  work  of  an  IPA,  and  based  on  that 
decision,  the  type  of  report  or  letter  they  will  issue.  The  degree  of  responsibility  varies 
by  type  of  report  or  letter  and  generally  increases  in  the  order  presented  below.  OIG 
auditors  may  decide  to: 

•  not  associate  the  OIG  with  the  IPA’s  work,' 

•  issue  a  letter  transmitting  the  IPA’s  report  and  expressing  either  no  opinion  or 
negative  assurance  (i.e.,  found  no  instances  of  material  noncompliance  with 
applicable  auditing  standards)  on  the  IPA’s  work, 

•  issue  a  report  that  refers  to  the  IPA’s  report  and  indicates  a  division  of 
responsibilities  between  the  IPA  and  the  OIG, 

•  issue  a  report  that  expresses  concurrence  or  non-concurrence  with  the  IPA’s 
report,  or 

•  issue  a  report  that  does  not  mention  the  IPA’s  work,  thus  accepting  and 
representing  the  work  of  the  IPA  as  the  OIG’s  work. 

FAM  650  also  provides  that  OIG  auditors  should  develop  a  written  plan  for  reviewing, 
and  if  necessary,  testing  the  work  done  by  an  IPA.  This  plan  should  document  the  level 
of  review  (high,  moderate,  or  low)  the  auditors  believe  is  necessary  based  on  the  type  of 
report  or  letter  they  will  issue  and  their  evaluation  of  audit  risk,  line  item  materiality,  and 
the  IPA’s  independence,  objectivity,  qualifications,  and  history.  The  level  of  review 
increases  as  the  degree  of  responsibility  for  the  IPA’s  work,  risk,  and  materiality 
increases  and  their  confidence  about  the  IPA’s  objectivity,  qualifications,  and 
performance  decreases. 

An  OIG  may  also  provide  representation  on  an  audit  committee  overseeing  the  work  of 
a  contracted  IPA.  In  so  doing,  the  OIG  participates  as  an  observer  or  acts  as  an  expert  in 
a  purely  advisory,  nonvoting  capacity  to  advise  entity  management  on  issues  based  on  the 
specialized  knowledge  and  skills  of  the  OIG  auditors. 


2  In  this  situation,  the  IPA’s  report  is  provided  directly  to  the  auditee  and  significant  users.  OIG  auditors  may  use 
this  method  when  the  OIG  merely  procures  the  audit  but  is  not  acting  as  “the  auditor.”  However,  the  contracting 
and  contract  oversight  process  generally  will  require  the  OIG  auditors  to  evaluate  the  IPA’s  independence, 
objectivity,  and  qualifications  and  to  monitor  performance  under  the  contract. 

3  In  the  DoD,  an  audit  committee  may  also  be  referred  to  as  a  “financial  audit  advisory  committee”  or  an  “audit 
preparedness  committee.” 
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Service  Contracting  Policies 

Management  Oversight  of  Service  Contracting.  Federal  Acquisition 
Regulation  37.102  requires  agencies  to  establish  effective  management  practices  in 
accordance  with  Office  of  Federal  Procurement  Policy  Letter  93-1,  “Management 
Oversight  of  Service  Contracting,”  to  prevent  fraud,  waste,  and  abuse  in  service 
contracting.  The  policy  letter  makes  agency  heads  responsible  for  ensuring  that  service 
contracts  are  awarded  and  administered  in  such  a  manner  so  that  customers  receive 
quality  services  on  time  and  within  budget.  It  also  emphasizes  the  use  of  “best  practices” 
techniques  when  contracting  for  services  and  in  contract  management  and  administration 
to  help  achieve  excellence  in  contractor  performance  and  offers  guidance  to  ensure  that 
agencies  use  good  management  practices  and  contract  administration  techniques  so  that: 

•  sufficient  resources  are  at  hand  to  evaluate  contractor  perfonnance  when  the 
statement  of  work  requires  the  contractor  to  provide  advice,  analysis  and 
evaluation,  opinions,  alternatives,  or  recommendations  that  could  significantly 
influence  agency  policy  development  or  decision-making; 

•  quality  assurance  plans  contain  enough  specifics  to  adequately  monitor  contractor 
performance; 

•  statements  of  work  specify  contract  deliverables  and  require  progress  reporting  on 
contractor  performance;  and 

•  adequate  expertise  exists  within  the  agency  to  independently  evaluate  the 
contractor’s  approach,  methodology,  results,  options,  conclusions,  or 

rec  ommendations . 

Contract  Quality  Assurance.  The  Federal  Acquisition  Regulation  Subparts  37. 1 
and  37.6  emphasize  the  use  of  performance  standards  and  quality  requirements  to  ensure 
that  appropriate  quality  levels  are  achieved  out  of  the  services  acquired  by  contract. 
Federal  Acquisition  Regulation  37.6  underscores  the  need  to  prepare  statements  of  work 
that  enable  the  assessment  of  work  performance  against  measurable  performance 
standards  and  quality  assurance  surveillance  plans  that  specify  all  work  requiring 
surveillance  and  the  methods  of  surveillance. 
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Policies  on  Services  by  Non-Federal  Auditors 

Inspector  General  Oversight  of  Non-Federal  Auditors.  The  Inspector  General  Act  of 
1978,  as  amended,  establishes  the  duties,  responsibilities,  and  authorities  of  the  Office  of 
the  Inspector  General  of  the  Department  of  Defense.  The  Act  gives  the  Inspector  General 
oversight  of  non-Federal  auditors  and  makes  the  Inspector  General  responsible  for 
establishing  guidelines  on  when  it  shall  be  appropriate  to  use  non-Federal  auditors  and 
taking  appropriate  steps  to  assure  that  any  work  perfonned  by  non-Federal  auditors 
complies  with  the  Government  Auditing  Standards  established  by  the  Comptroller 
General  of  the  United  States  for  audits  of  Federal  establishments,  organizations, 
programs,  activities,  and  functions. 

DoD  Contracting  for  Audit  Services.  The  DoD  OIG  established  guidelines  on  the  use 
of  non-Federal  auditors  in  DoD  Instruction  7600.02,  “Audit  Policies.”  The  instruction 
pennits  DoD  Components  to  contract  for  audit  services  when  applicable  expertise  is 
unavailable  within  the  DoD  audit  organization,  augmentation  of  the  DoD  audit 
organization’s  audit  staff  is  necessary  to  execute  the  annual  audit  plan,  or  temporary  audit 
assistance  is  required  to  meet  audit  reporting  requirements  mandated  by  law  or  DoD 
Regulation.  Components  are  to  obtain  a  review  of  the  solicitation  for  audit  services  from 
the  Office  of  the  Assistant  Inspector  General  for  Audit  Policy  and  Oversight,  DoD  OIG, 
before  releasing  the  solicitation  to  prospective  contractors.  The  purpose  of  this  review  is 
to  ensure  that  the  use  of  non-Federal  auditors  is  in  compliance  with  DoD 
Instruction  7600.02  and  applicable  laws  and  regulations  and  that  the  solicitation  requires 
compliance  with  applicable  auditing  standards. 

Compliance  with  Audit  Standards.  Defense  Federal  Acquisition  Regulation 
Supplement  237.270  requires  that  contracts  for  audit  services  include  a  clause  requiring 
the  contractor,  in  the  performance  of  all  audit  services  under  the  contract,  to  comply  with 
Government  Auditing  Standards  issued  by  the  Comptroller  General  of  the  United  States. 
These  standards  provide  a  framework  for  high-quality  audit  work  by  requiring  audit 
organizations  to  (1)  maintain  the  highest  degree  of  objectivity  and  independence  when 
performing  its  work;  (2)  have  staff  with  the  appropriate  technical  knowledge,  skills,  and 
experience  conduct  the  work;  (3)  maintain  a  system  of  quality  control  for  ensuring 
compliance  with  applicable  auditing  standards;  and  (4)  periodically  undergo  an  external 
peer  review  of  its  quality  control  system.  The  standards  require  auditors  to  adequately 
plan  their  work  and  to  obtain  sufficient  appropriate  evidence  to  support  their  conclusions 
and  opinions.  Also,  a  written  record  of  the  auditors’  evidence  must  be  retained  in  the 
form  of  audit  documentation  and  should  show  the  work  performed,  evidence  obtained, 
and  conclusions  reached  as  well  as  adequate  planning  and  proper  supervision. 
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Results  of  Review 


DoD  Components  contracting  for  audit  services  and  the  DoD  OIG  have  a  shared  goal  of 
reasonable  assurance  of  contractor  compliance  with  Government  Auditing  Standards  in 
the  perfonnance  of  audit  work  and  the  reporting  of  audit  results.  With  this  goal  in  mind, 
we  perfonned  this  review  in  part  to  examine  how  DoD  Components  manage  and  conduct 
oversight  of  contracted  audit  services  to  ensure  contractors  not  only  meet  contract 
requirements  but  also  follow  applicable  auditing  standards.  From  the  detailed  review  of 
five  contracts,  we  learned  that  oversight  capabilities  and  practices  varied  and  that  certain 
key  strategies  need  to  be  disseminated.  This  report  summarizes  what  we  observed  and 
suggests  the  following  strategies  that  we  believe  are  essential  to  effective  and  efficient 
oversight  of  contracted  audit  services. 

•  Designate  qualified  personnel  with  significant  experience  in  auditing  and 
appropriate  training  in  contract  administration  as  contracting  officers’ 
representatives  (CORs). 

•  Prepare  a  comprehensive  written  plan  for  overseeing  contracted  audit  services. 

•  Monitor  and  evaluate  the  contractor’s  work  as  it  progresses  to  resolve  problems 
that  may  result  in  reduced  audit  quality4,  missed  deadlines,  or  additional  costs. 

•  Monitor  contractor  independence  under  the  standards  for  each  audit  perfonned 
and  secure  contractor  resolution  of  any  independence  impairment  identified. 

•  Monitor  the  adequacy  of  the  contractor’s  quality  control  system  and  the 
competence  of  its  staff  and  address  conditions  that  may  jeopardize  audit  quality. 

Other  Federal  Offices  of  Inspectors  General  (OIGs)  also  contract  for  audit  services  and 
share  the  common  goal  of  receiving  quality  services  on  time,  within  cost,  and  compliant 
with  applicable  auditing  standards  and  the  contract.  Therefore,  we  surveyed  selected 
OIGs  about  their  contract  oversight  practices.  Key  practices  included: 

•  communicating  expectations  and  results, 

•  establishing  detailed  milestones, 

•  tracking  deliverables,  and 

•  determining  lessons  learned. 


4  Audit  quality  as  used  in  this  report  refers  to  the  auditor’s  compliance  with  applicable  auditing  standards.  Auditing 
standards  are  generally  accepted  measures  of  audit  quality.  Compliance  with  standards  helps  assure  users  of  audit 
reports  that  the  auditor  has  adequately  performed  the  audit  and  that  the  audit  report  can  be  relied  upon. 
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COR  Qualifications 


Designating  CORs  with  significant  auditing  experience  and  appropriate  contract 
administration  training  is  essential  for  effective  oversight  of  contracted  audit  services. 
Defense  Federal  Acquisition  Regulation  Supplement  201.602-2  requires  that  the  COR  be 
qualified  by  training  and  experience  commensurate  with  the  specific  technical  monitoring 
and  administrative  responsibilities  delegated  to  them.  Accordingly,  CORs  designated  to 
perform  technical  monitoring  and  administration  of  a  contract  for  “audit  services”  should 
be  qualified  by  training  and  experience  in  auditing  and  contract  administration. 

Preferably  this  would  be  the  same  competencies  and  skills  as  experienced  auditors  who 
perform  audit  work  in  accordance  with  Government  Auditing  Standards,5  as  well  as 
training  on  contract  administration  functions  such  as  proper  file  documentation  and 
performance  assessment  methods.  Having  personnel  with  those  attributes  as  CORs  is 
desirable  and  helps  establish  a  foundation  for  effective  oversight. 

The  following  table  summarizes  what  we  observed  with  regard  to  the  experience  and 
training  of  the  CORs  designated  to  oversee  the  selected  contracts  for  audit  services.  The 
first  column  identifies  desirable  but  not  mandatory  attributes  in  a  COR.  Also,  some  of 
the  attributes  are  more  relevant  to  certain  types  of  audits  than  others.  For  example,  the 
professional  designation  as  a  certified  public  accountant  is  an  attribute  more  relevant  to 
the  qualifications  necessary  in  overseeing  a  financial  statement  audit  than  an  information 
assurance  audit.6 


5  Government  Auditing  Standards  define  an  “experienced  auditor”  as  an  individual  who  possesses  the  competencies 
and  skills  that  would  have  enabled  him  or  her  to  perform  the  audit,  which  includes  an  understanding  of  (1)  audit 
processes,  (2)  Government  Auditing  Standards  and  applicable  legal  and  regulatory  requirements,  (3)  the 
environment  in  which  the  audited  entity  operates,  and  (4)  issues  relevant  to  the  audited  entity’s  environment. 

6  An  information  assurance  audit  (also  known  as  a  SAS  70/88  review)  involves  examining  a  service  organization’s 
controls  over  transactions  processing,  including  information  systems  controls,  and  is  generally  designed  to  provide 
user  organizations  and  their  auditors  with  information  about  the  service  organization’s  internal  control  environment. 
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Table  1.  Qualifications  of  CORs  Designated  on  Selected  Contracts 


DeCA 

DFAS 

MERHCF 

DDRS 

DISA 

Attribute  FS  Audit 

FS  Audit  FS  Audit 

IA  Audit 

IA  Audit 

Certificate  of  Training  for  COR  Course 

• 

• 

• 

• 

• 

Certified  Information  Systems  Auditor 

• 

• 

Certified  Public  Accountant 

• 

• 

Entity  Specific  Accounting  Experience 

• 

• 

Entity  Specific  Auditing  Experience 

• 

• 

• 

Senior-Level  Accounting  Experience 

• 

• 

Senior-Level  Auditing  Experience 

• 

• 

• 

DeCA  Defense  Commissary  Agency 

DDRS  Defense  Departmental  Reporting  System 
DFAS  Defense  Finance  and  Accounting  Service 

FS 

IA 

MERHCF 

Financial  Statements 

Information  Assurance 
Medicare-Eligible  Retiree  Health 

DISA  Defense  Information  Systems  Agency 

Care  Fund 

Held  a  senior-level  position  such  as  partner,  director,  manager,  senior  auditor,  or  senior  accountant. 


Two  of  the  contracts  had  CORs  that  did  not  have  auditing  experience  beyond  three  years 
and  the  CORs  did  not  specifically  engage  an  audit  specialist  to  assess  audit  quality  or 
contractor  quality  control  before  the  contractors  issued  their  draft  audit  reports.  This 
condition  could  jeopardize  the  effectiveness  of  the  contract  oversight  because  the  CORs 
alone  may  not  have  enough  practical  experience  in  performing  and  reviewing  audit  work 
to  be  technically  competent  in  spotting  deficiencies  in  the  contractors’  work  early  on  and 
as  it  progresses.  Because  limited  guidance  exists  as  to  the  specific  attributes  necessary  in 
a  COR,  we  encourage  the  use  of  GAO/PCIE  FAM  Section  650  as  a  guide  in  designating 
CORs  on  audit  services  contracts.  FAM  650  basically  indicates  that  staff  reviewing  the 
work  of  auditors  under  contract  generally  should  have  enough  experience  in  the  type  of 
audit  being  performed  to  understand  the  judgments  that  need  to  be  made  by  the  auditors 
and  to  interact  with  the  higher  levels  of  the  auditors’  organization.  Further,  most  of  the 
review  generally  should  be  done  by  or  under  the  direction  of  an  assistant  director  or  a 
manager  who  has  significant  experience  in  perfonning  and  reviewing  the  type  of  audit 
work  being  perfonned. 


We  suggest  that  components  without  CORs  having  significant 
experience  in  performing  and  reviewing  audit  work  of  the  type 
performed,  seek  the  assistance  of  other  government  personnel  with  the 
necessary  technical  expertise,  and  have  them  assess  audit  quality  and 
contractor  quality  control. 
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Oversight  Planning 


Preparing  a  comprehensive  written  plan  for  overseeing  contracted  audit  services  is 
essential  for  effective  and  efficient  oversight. 


Plan 

Prioritize  &  Focus 
^Level  of  Review 

v" Oversight  Strategies 

^Monitoring  Procedures 


A  well-conceived  plan  documents  the  level  of  review,  oversight  strategies,  and 
monitoring  procedures  the  COR  detennines  is  necessary  to  ensure  that  the  contractor 
delivers  quality  services  on  time,  within  cost,  and  compliant  with  applicable  auditing 
standards  and  the  contract.  The  plan  also  documents  the  COR’s  assessments  of  the 
contractor’s  objectivity,  qualifications,  past  performance,  and  system  of  quality  control, 
as  well  as  assessments  of  risk  and  materiality  such  as  technical,  cost,  and  schedule  risks;7 8 

Q 

inherent  and  control  risk  conditions;  and  matters  individually  or  collectively  significant 
to  the  audited  entity  or  primary  users  of  the  audited  information.9 


7  Technical,  cost,  and  schedule  risks  are  potential  problem  areas  posing  the  greatest  risk  to  the  contractor’s  ability  to 
meet  contract  requirements  and  deliver  quality  services  on  time  and  within  cost. 

8  Inherent  and  control  risk  conditions  are,  for  example,  areas  with  a  history  of  significant  audit  adjustments,  new 
types  of  transactions,  transactions  or  accounts  subject  to  significant  management  judgments  (e.g.,  estimates),  new  or 
significantly  changed  information  systems,  and  areas  with  known  control  deficiencies. 

9  Significant  matters  include,  for  example,  significant  line  items,  individual  account  balances,  or  classes  of 
transactions,  or  components  individually  generating  transactions  or  account  balances  at  a  level  significant  to  the 
overall  entity. 
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The  level  of  review  is  a  judgment  the  COR  makes  and  generally  increases  as  risk  and 
materiality  increases  and  the  COR’s  confidence  in  the  contractor’s  objectivity, 
qualifications,  performance,  and  quality  control  decreases. 


Level  of  Review 


The  strategies  and  procedures  to  be  carried  out  are  tailored  to  the  planned  level  of  review 
and  include  steps  to  assess  audit  quality  and  the  contractor’s  quality  control.  Having 
plans  designed  this  way  is  desirable  and  helps  establish  a  foundation  for  effective  and 
efficient  oversight  by  helping  CORs  prioritize  and  focus  their  oversight  efforts  on  areas 
of  higher  risk  and  materiality. 

The  following  table  summarizes  what  we  observed  with  regard  to  the  oversight  planning 
that  the  CORs  on  the  selected  contracts  performed.  The  first  column  identifies  desirable 
but  not  mandatory  planning  actions. 


Table  2.  Oversight  Planning  Observed  on  Selected  Contracts 

DeCA  DFAS  MERHCF  DDRS  DISA 
Planning  Action  FS  Audit  FS  Audit  FS  Audit  IA  Audit  IA  Audit 

Documented  Level  of  Review  -  L,  M,  FI  •  •  • 

Documented  Basis  for  Level  of  Review  •  •  • 

Documented  Strategies,  Monitoring  Plans  •  •  •  •  • 

Documented  Steps  to  Assess  Audit  Quality  •  •  • 

Documented  Steps  to  Assess  Quality  Control  •  •  • 

Scheduled  Tasks  and  Task  Milestones  •  • 

Scheduled  Deliverables  and  Delivery  Dates  •  •  •  •  • 

*  L  (low),  M  (moderate),  FI  (high) 


The  CORs  on  all  five  contracts  took  steps  to  identify  the  contractors’  work  plans  and 
schedules  and  also  prepared  written  plans  of  action  or  schedules  for  use  in  monitoring  the 
contractors’  performance.  However,  none  of  the  documented  planning  actions  indicated 
that  the  CORs  based  their  strategies,  techniques,  and  procedures  on  factors  of  risk, 
materiality,  and  contractor  traits.  For  example,  it  was  not  evident  how  the  CORs 
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analyzed  and  used  factors  such  as  the  ones  listed  in  the  following  figure  to  detennine 
their  plan  of  what  needed  to  be  monitored,  how,  and  how  often. 


Factors  in  Determining  a  Contract  Monitoring  Plan 

■S  Technical  complexity  of  the  contract  tasks 
•S  Degree  of  cost  monitoring  needed 
•S  Urgency  of  the  delivery  schedule 

•S  Nature  and  extent  of  inherent  risk  conditions  in  the  areas  under  audit 
■S  Existence  of  known  control  weaknesses  in  the  areas  under  audit 
•S  Materiality  or  significance  of  the  areas,  items,  or  sites  under  audit 
■S  Objectivity  of  the  contractor  and  its  staff  with  respect  to  the  audited  entity 
•S  Knowledge,  skills,  and  experience  of  the  contractor’s  staff  for  the  tasks  assigned 
•S  Contractor’s  past  performance  as  to  quality,  timeliness,  and  cost  control 
•S  Nature  and  extent  of  the  contractor’s  quality  control  policies  and  procedures 
■S  Results  of  the  latest  external  peer  review  of  the  contractor’s  quality  control  system 


Also,  the  CORs  on  two  contracts  did  not  include  in  their  plans  steps  to  assess  audit 
quality  or  the  extent  of  the  contractor’s  quality  control  methods.  An  example  of  a  step 
would  be  to  identify  the  contractor’s  methods  of  controlling  audit  quality,  such  as 
supervisory,  technical,  and  other  quality  control  review,10  and  determine  whether  a 
selection  of  the  contractor’s  key  audit  documentation11  complies  with  applicable  auditing 
standards  and  other  relevant  requirements  and  evidences  appropriate  quality  control 
review. 


We  encourage  CORs  overseeing  DoD  contracted  audit  services  to 
discuss  in  their  written  oversight  plans  their  judgments  about  specific 
risks,  areas  of  significance,  and  characteristics  of  the  contractor.  CORs 
should  determine  the  level  of  review,  oversight  strategies,  and 
monitoring  procedures  necessary  for  effective  and  efficient  oversight. 
Including  determinations  on  monitoring  needs  (how  and  how  often)  and 
procedures  to  assess  audit  quality  and  contractor  quality  control. 


10  Supervisory,  technical,  and  other  quality  control  review  includes,  for  example,  supervisory  reviews  by  appropriate 
first  line  and  second  line  supervisors;  technical  reviews  by  specialists  such  as  actuaries,  information  technology 
experts,  and  statisticians;  and  quality  control  reviews  by  individuals  independent  of  the  audit  engagement  such  as 
concurring  review  partners  and  report  referencing  reviewers. 

1 1  Key  audit  documents  are,  for  example,  audit  plans,  work  programs,  working  papers  concerning  conclusions  on 
principal  audit  areas,  high-risk  matters,  or  major  issues,  working  papers  supporting  matters  reported,  and  draft  and 
final  audit  reports. 
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Audit  Monitoring 


Monitoring  and  evaluating  the  contractor’s  work  as  it  progresses  is  essential  to  effective 
oversight  of  contracted  audit  services. 


Monitor  &  Evaluate 
Contractor  Work 


Monitoring  and  evaluating  the  contractor’s  work  helps  to  identify  early  signs  of  any 
performance  problems  that  may  result  in  reduced  audit  quality,  missed  deadlines,  or 
additional  costs  if  not  resolved.  Monitoring  can  be  accomplished  by  meetings  with  the 
contractor  early  on  about  the  scope  and  timelines  of  the  audit  work,  obtaining  progress  or 
status  reports  from  the  contractor,  and  participating  in  key  meetings  “  between  the 
contractor  and  entity  officials.  Evaluating  performance  can  be  accomplished  by 
reviewing  the  contractor’s  plans,  key  documentation,  and  written  products  or  deliverables 
for  technical  completeness,  evidence  of  quality  control  review,  and  compliance  with 
applicable  auditing  standards  and  the  contract.  Documenting  actions  taken  in  overseeing 
a  contract  is  also  important.  Defense  Federal  Acquisition  Regulation  Supplement 
Procedures,  Guidance,  and  Information  201.602-2  requires  that  the  COR  maintain  a  file 
containing  documentation  of  their  actions  in  accordance  with  the  authority  and 
responsibilities  delegated  to  them  by  the  contracting  officer.  Documentation  can  take 
many  forms,  and  when  done  thoroughly,  can  be  very  useful  in  the  event  of  contract 
dispute  by  providing  a  complete  picture  of  discussions  held,  actions  taken,  problems 
identified,  and  decisions  made  by  the  COR  as  well  as  any  actions  by  the  contractor  in 
response  to  COR  requests. 


p  Key  meetings  are,  for  example,  opening  and  completion  conferences,  planning  meetings,  meetings  discussing 
high-risk  or  significant  areas,  and  meetings  discussing  the  contractor’s  conclusions. 
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The  following  table  summarizes  what  we  observed  with  regard  to  the  audit  monitoring 
that  the  CORs  on  the  selected  contracts  perfonned.  The  first  column  identifies  desirable 
but  not  mandatory  monitoring  actions.  In  addition,  some  of  the  monitoring  actions  are 
more  relevant  to  certain  types  of  audits  than  others.  For  example,  reviewing  the 
contractor’s  account  risk  analysis  is  a  monitoring  action  relevant  to  overseeing  a  financial 
statement  audit  not  an  information  assurance  audit.  Also,  the  nature,  timing,  and  extent 
of  the  monitoring  actions  performed  can  vary  depending  on  the  facts  and  circumstances 
of  each  contract  or  audit  situation  and  the  level  of  review  the  COR  determines  is 
necessary.  For  example,  monitoring  actions  can  be  more  extensive  during  the  first  year 
with  a  new  contract  or  new  contractor  as  well  as  during  the  audit  of  a  particular  unit  or 
segment(s)  of  an  organization  that  is  unique  or  complex  and  less  extensive  when  the  COR 
is  thoroughly  familiar  with  the  contractor’s  quality  control  methods  and  decides  to  rely 
on  them. 


Table  3.  Audit  Monitoring  Observed  on  Selected  Contracts 


DeCA3  DFAS3  MERHCF  DDRS  DISA 
Monitoring  Action  FS  Audit  FS  Audit  FS  Audit  1A  Audit  1A  Audit 


Attended  Entrance  and  Exit  Conferences 

• 

• 

• 

• 

• 

Attended  Planning  Meetings 

• 

• 

• 

• 

• 

Attended  Key  Meetings  Field  By  Contractor 

• 

• 

• 

• 

• 

Obtained  Periodic  Progress  or  Status  Reports 

• 

• 

• 

• 

• 

Verified  Deliverables  Met  Contract  Terms 

• 

• 

• 

• 

• 

Identified  QCs1  for  Ensuring  Audit  Quality 

• 

• 

• 

Verified  Whether  QCs  In  Use 

• 

• 

• 

Reviewed  key  audit  documentation,  e.g.: 


-  data,  document,  sampled  items  request  lists 

• 

• 

-  notices  of  findings 

• 

• 

-  audit  plans 

• 

• 

• 

-  client  (or  entity)  profile  • 

-  general  risk  analysis 

• 

• 

• 

-  account  risk  analysis  • 

-  cycle  memorandums,  flowcharts 

• 

• 

• 

-  specific  control  evaluations 

• 

• 

• 

-  test  procedures/plans 

• 

• 

• 

-  summary  memorandums  of  work  and  results  •  • 

-  draft  audit  reports 

• 

• 

• 

• 

• 

-  final  audit  reports 

• 

• 

• 

• 

• 

-  report  cross-references  to  supporting  WPs2 3 

• 

• 

• 

1  QCs  (quality  controls)  such  as  supervisory,  technical,  and  other  quality  control  types  of  review. 

2  WPs  (working  papers)  or  documentation  of  work  performed,  evidence  obtained,  conclusions  reached. 

3  CORs  had  two  or  more  years  experience  with  the  same  contractor  and  relied  on  the  contractor’s  most 
recent  peer  review  report  in  which  the  external,  independent  peer  expressed  an  unmodified  opinion  on  its 
quality  control  system  for  ensuring  compliance  with  applicable  standards  in  the  conduct  of  its  work. 
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The  CORs  on  all  five  contracts  successfully  monitored  the  progress  of  the  contractors’ 
audit  work  as  well  as  the  contractors’  compliance  with  contract  terms  and  conditions  so 
that  the  contractors  accomplished  the  deliverables  called  for  in  the  contracts.  However, 
we  noted  that  the  CORs  on  two  contracts  had  limited  awareness  of  resources  that  could 
assist  them  in  designing  and  perfonning  their  monitoring  procedures  and  may  benefit 
from  learning  about  them.  Namely,  GAO/PCIE  FAM  Section  650  and  the  monitoring 
tool  developed  by  the  Federal  Audit  Executive  Council  (FAEC),  a  subcommittee  of  the 
President’s  Council  on  Integrity  and  Efficiency.  FAM  650  illustrates  the  procedures  that 
generally  should  be  performed  for  high,  moderate,  and  low  levels  of  review  of  contracted 
audit  work,  as  well  as  what  generally  should  be  retained  as  documentation  of  the 
procedures  performed.  The  FAEC  tool  incorporates  best  practices  and  lessons  learned  of 
Federal  OIGs  into  a  detailed  program  for  use  in  monitoring  contracted  financial  statement 
audits. 


We  encourage  CORs  overseeing  DoD  contracted  audit  services  to  utilize 
FAM  650  and  the  FAEC  tool  as  resources  in  designing  and  performing 
their  oversight.  FAM  650  can  be  found  on  the  U.S.  Government 
Accountability  Office  website  at  www.sao.sov  and  the  FAEC  tool  can  be 
found  on  the  IGNet  website  for  Federal  OIGs  at 
www.  isnet.sov/pande/faecl.  html. 


Monitoring  Contractor  Independence 


Users  of  audit  work  done  in  accordance  with  Government  Auditing  Standards  should 
have  confidence  that  the  work  is  objective.  For  this  reason,  Government  Auditing 
Standards  require  that,  in  all  matters  relating  to  the  audit  work,  the  audit  organization  and 
the  individual  auditor  be  free  from  personal,1’  external,14  and  organizational15 
impairments  to  independence  and  avoid  the  appearance  of  such  impairments.  The  audit 
organization  and  individual  auditor  must  maintain  their  independence  so  that  their 
judgments  on  all  issues  associated  with  conducting  and  reporting  on  the  audit  work  are 
impartial  and  viewed  as  impartial  by  knowledgeable  third  parties. 


13  Personal  impairments  of  auditor  independence  result  from  personal  relationships  or  beliefs  that  might  cause  the 
auditor  to  limit  the  extent  of  the  inquiry,  limit  disclosure,  or  weaken  or  slant  audit  findings. 

14  External  impairments  to  independence  result  from  factors  external  to  the  audit  organization,  such  as  pressures 
(actual  or  perceived)  from  management  and  employees  of  the  audited  entity,  that  may  restrict  the  audit  work  or  deter 
the  auditors  from  acting  objectively  and  exercising  professional  skepticism. 

15  Organizational  impairments  to  independence  result  when  the  audit  organization  provides  nonaudit  services 
(i.e.,  services  not  performed  in  accordance  with  Government  Auditing  Standards)  directly  supporting  an  entity’s 
operations  thus  impairing  its  ability  to  meet  the  overarching  independence  principles  that  audit  organizations  must 
not  (1)  perform  management  functions  or  make  management  decisions  and  (2)  audit  their  own  work  or  provide 
nonaudit  services  significantly  or  materially  affecting  the  subject  matter  of  the  audits. 
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Monitoring  the  contractor’s  independence  under  the  standards  for  each  audit  performed 
and  securing  contractor  resolution  of  any  impairment  to  independence  is  essential  to 
effective  oversight  of  contracted  audit  services.  User  confidence  in  the  results  of  the 
audit  work  is  strengthened  knowing  the  contractor  followed  the  independence  standards 
and  avoided  conflicts  that  may  impair  its  objectivity.  Monitoring  contractor 
independence  for  each  audit  performed  can  be  accomplished  by  obtaining  representation 
from  the  contractor  that  it  is  not  impaired  by  external  factors  or  any  past,  ongoing,  or 
planned  work  involving  the  audited  entity  and  that  its  audit  staff  are  free  of  personal 
impairments  to  independence.  The  representation  covers  the  period  of  the  activities 
under  audit  to  the  date  of  the  audit  report.  If  any  actual  or  apparent  impairment  exists, 
CORs  should  ask  the  contractor  to  demonstrate  that  it  has  resolved  the  impairment,  for 
example,  by  substituting  staff.  Also,  CORs  should  become  familiar  with  the  contractor’s 
process  for  ensuring  independence  under  the  standards  and  determine  whether  the 
contractor  followed  that  process  for  the  audit.  Doing  so  helps  to  determine  the  extent  to 
which  the  independence  representations  of  the  contractor  can  be  relied  upon. 

The  following  table  summarizes  what  we  observed  with  regard  to  the  actions  that  the 
CORs  on  the  selected  contracts  took  to  monitor  the  independence  of  the  contractors.  The 
first  column  identifies  desirable  but  not  mandatory  monitoring  actions. 


Table  4.  Monitoring  of  Contractor  Independence  on  Selected  Contracts 

Monitoring  Action 

Determined  Firm  Independence  For  Audit 

DeCA 

FS  Audit 

DFAS 

FS  Audit 

MERHCF 
FS  Audit 

• 

DDRS 

IA  Audit 

• 

DISA 

IA  Audit 

• 

Determined  Staff  Independence  For  Audit 

• 

• 

• 

Determined  Independence  of  New  Staff 

n/a* 

n/a 

Identified  Process  for  Ensuring  Independence 

• 

• 

• 

• 

Verified  If  Independence  Process  In  Use 

*  n/a  (not  applicable)  because  no  one  joined  or 

replaced  members  of  the  initial  audit  team. 

Not  all  of  the  CORs  determined  firm  independence  and  staff  independence  for  each  audit 
performed  or  detennined  the  independence  of  staff  joining  or  replacing  members  of  the 
initial  audit  team.  Specifically,  the  CORs  on  two  contracts  did  not  obtain  any  contractor 
representations  as  to  independence  and  the  CORs  on  two  other  contracts  did  not 
determine  the  independence  of  new  staff.  In  one  case  this  occurred  because  the  CORs 
relied  on  the  contractors’  self-reporting  of  independence  issues  in  accordance  with 
contractual  language  requiring  the  contractors  to  be  independent  as  defined  in  standards, 
as  well  as  clauses  precluding  them  from  engaging  in  work  that  could  be  a  conflict  of 
interest.  Also,  not  one  of  the  CORs  on  the  five  contracts  verified  whether  the 
contractors’  actually  put  in  use  a  process  for  ensuring  independence  under  the  standards. 
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We  encourage  CORs  overseeing  DoD  contracted  audit  services  to  identify 
and  document  for  each  audit  the  contractor’s  organizational  independence 
and  the  independence  of  the  individual  engagement  team  members.  CORs 
should  identify  and  document  the  independence  of  initial  and  new  staff, 
and  verify  the  nature  and  extent  of  the  contractor’s  methods  to  maintain 
independence. 


Monitoring  Contractor  Qualifications 


Monitoring  the  adequacy  of  the  contractor’s  quality  control  system  and  the  competence 
of  its  audit  staff  and  addressing  conditions  that  may  jeopardize  audit  quality  is  essential  to 
effective  oversight  of  contracted  audit  services.  User  confidence  in  the  quality  of  the 
audit  work  and  reported  results  is  strengthened  knowing  the  contractor  followed 
Government  Auditing  Standards  on  quality  control  and  staff  competence16  and  had  its 
system  of  quality  control  reviewed  as  well  as  qualified  staff  perform  the  work. 

Monitoring  contractor  qualifications  can  be  accomplished  by  reviewing  the  contractor’s 
most  recent  peer  review  report,  the  related  letter  of  comments,  and  the  contractor’s 
response  to  the  review  report.  CORs  should  adjust  the  level  of  their  oversight  as 
appropriate  based  on  the  significance  of  any  changes  to  the  contractor’s  quality  control 
policies  and  procedures  since  the  peer  review  and  any  remaining  weaknesses  in  the 
system. 

Monitoring  staff  qualifications  can  be  accomplished  by  reviewing  resumes  to  determine 
whether  the  staff  has  the  experience  necessary  for  the  work  such  as  government  auditing 
experience,  experience  in  the  type  of  audit  work,  or  experience  with  audits  of  similar 
entities,  as  well  as  educational  and  professional  qualifications  appropriate  for  their  role. 

If  any  deficiency  in  staff  expertise  is  identified,  CORs  may  require  the  contractor  to 
substitute  more  qualified  staff  to  ensure  the  audit  is  performed  by  staff  who  collectively 
have  the  knowledge,  skills,  and  experience  necessary  for  that  assignment. 

The  following  table  summarizes  what  we  observed  with  regard  to  the  monitoring  of 
contractor  qualifications  that  the  CORs  on  the  selected  contracts  performed.  The  first 
column  identifies  desirable  but  not  mandatory  monitoring  actions. 


16  Government  Auditing  Standards  require  audit  organizations  to  have  its  system  of  quality  control  reviewed  by  an 
external,  independent  peer  at  least  once  every  3  years  and  to  ensure  each  audit  is  performed  by  a  staff  member  or 
team  of  staff  members  with  sufficient  and  appropriate  technical  knowledge,  skills,  and  experience  for  the  work. 
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Table  5.  Monitoring  of  Contractor  Qualifications  on  Selected  Contracts 


Monitoring  Action 

Reviewed  Most  Recent  Peer  Review  Report 

DeCA 

FS  Audit 

• 

DFAS 

FS  Audit 

• 

MERHCF 
FS  Audit 

• 

DDRS 

IA  Audit 

• 

DISA 

IA  Audit 

• 

Identified  QC1  Changes  Since  Review 

2 

2 

• 

• 

• 

Adjusted  Oversight  If  QCs  Deficient 

2 

2 

3 

3 

3 

Reviewed  Resumes  of  Managers  and  Partners 

• 

• 

• 

• 

• 

Reviewed  Resumes  of  Senior  Auditors 

• 

• 

• 

Required  Staff  Changes  If  Jointly  Deficient 

4 

4 

4 

1  QC  (quality  control) 

2  CORs  had  two  or  more  years  experience  with  the  same  contractor  and  relied  on  the  contractor’s  most 
recent  peer  review  report  in  which  the  external,  independent  peer  expressed  an  unmodified  opinion  on  its 
quality  control  system  for  ensuring  compliance  with  applicable  standards  in  the  conduct  of  its  work. 

3  Not  applicable  because  the  most  recent  peer  review  report  presented  an  unmodified  opinion  on  the 
contractor’s  quality  control  system. 

4  Not  applicable  because  the  audit  staff  collectively  had  the  technical  knowledge,  skills,  and  experience 
necessary  for  the  audit  assignment. 


The  CORs  on  all  five  contracts  monitored  the  adequacy  of  the  contractor’s  quality  control 
system  by  reviewing  peer  review  reports  and  monitored  the  competence  of  audit 
management  by  reviewing  resumes.  However,  the  CORs  on  two  contracts  did  not  assess 
the  background  and  suitability  of  the  senior  auditors  for  the  audit  tasks,  particularly  those 
supervising  the  day-to-day  operations  of  the  audit,  directing  staff  auditors  in  the  work 
performed,  and  reviewing  the  work  of  staff. 


We  encourage  CORs  overseeing  DoD  contracted  audit  services  to  review 
the  resumes  of  all  supervisory  audit  staff,  including  partners,  managers, 
and  senior  auditors,  to  ensure  that  the  work  is  supervised  by  personnel 
with  the  qualifications  to  ensure  audit  quality. 
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Practices  of  Other  Federal  OIGs 


Other  Federal  offices  of  inspectors  general  also  contract  for  audit  services  and  share  the 
common  goal  of  receiving  quality  services  on  time,  within  cost,  and  compliant  with 
applicable  auditing  standards  and  the  contract.  Therefore,  we  wanted  to  find  out  about 
the  practices  of  other  Federal  OIGs.  We  judgmentally  selected  the  following  seven 
Federal  executive  departments  and  agencies  with  interagency  relationships  with  the  DoD 
and  surveyed  their  OIGs  about  the  OIGs’  use  of  audit  contractors  and  the  OIGs’  practices 
and  experiences  overseeing  audit  services  contracts. 

•  Department  of  Homeland  Security 

•  Department  of  the  Interior 

•  Department  of  Justice 

•  Department  of  State 

•  Department  of  the  Treasury 

•  Department  of  Veterans  Affairs 

•  National  Aeronautics  and  Space  Administration 

The  OIGs  shared  with  us  some  of  the  practices  they  employ  when  overseeing  contracts 
for  auditing  services.  Highlights  of  the  practices  include: 

•  communicating  expectations  and  results, 

•  establishing  detailed  milestones, 

•  tracking  deliverables,  and 

•  determining  lessons  learned. 

Communicating  Expectations  and  Results.  OIGs  stated  that  communication 
throughout  the  contract  period  was  important.  Communicating  expectations  at  the 
beginning  of  the  contract  through  a  well-defined  statement  of  work  was  critical.  They 
identified  that  it  is  also  important  that  the  contractors  communicate  their  results  timely. 
Several  of  the  OIGs  indicated  that  contractors  generally  communicated  the  status  of  the 
audit  during  bi-weekly  briefings  and  progress  reports. 

Establishing  Detailed  Milestones.  OIGs  stated  that  it  was  important  to  set  detailed 
milestones  in  the  statement  of  work.  This  provides  timelines  for  the  information  OIGs 
receive  from  contractor  for  the  OIGs  to  effectively  evaluate  and  make  decisions.  The 
milestones  help  to  both  get  the  necessary  information  in  a  timely  manner  and  to  prevent 
disputes  about  contract  requirements  and  due  dates  for  deliverables. 

Tracking  Deliverables.  OIGs  established  a  system  for  tracking  deliverables.  The 
tracking  of  the  deliverables  allows  the  OIG  to  determine  whether  the  contractor  is 
meeting  the  requirements  of  the  contract.  It  also  helps  to  determine  whether  the 
contractor  is  meeting  the  milestones  and  whether  the  OIGs  are  getting  the  type  of  product 
that  is  required  in  the  contract. 
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Determining  Lessons  Learned.  OIGs  sought  out  a  process  for  identifying  lessons 
learned  from  contracting  for  auditing  services.  One  of  the  OIGs  had  meetings  with  the 
contractor  and  prepared  a  lessons  learned  about  the  audit.  This  process  allowed  for  both 
the  OIGs  and  the  contractors  to  comment  about  the  process. 


Conclusion 


We  recognize  that  limited  guidance  exists  as  to  specific  strategies,  techniques,  and 
procedures  for  overseeing  contracted  audit  services.  Therefore,  this  report  contains  no 
recommendations.  Instead,  it  identifies  key  strategies  and  practices  we  believe  are 
essential  to  effective  and  efficient  oversight.  This  report  is  intended  for  use  by  DoD 
contracting  and  contract  oversight  officials.  While  the  suggestions  presented  are  not 
mandatory,  we  hope  that  employing  them  will  guide  the  DoD  officials  towards  taking 
appropriate  steps  to  monitor  and  evaluate  contractor  performance  early  on  and  as  the 
audit  progresses,  thereby  addressing  and  resolving  problems  that  may  result  in  reduced 
audit  quality,  missed  deadlines,  or  additional  costs  before  the  audit  is  completed.  Doing 
so  will  help  ensure  that  the  DoD  entities  contracting  for  or  requesting  audit  services 
receive  quality,  timely  audit  results  that  can  be  relied  upon  for  decision  making  purposes. 
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Appendix  A.  Scope  and  Methodology 


This  review  was  self-initiated.  The  overall  review  objective  was  to  determine  the 
effectiveness  of  and  lessons  learned  from  the  oversight  of  DoD  contracted  audit  services. 
We  detennined  the  extent  of  DoD’s  use  of  contractors  for  performing  audit  services. 
Analysis  of  DoD  contracting  activity  for  FYs  2004  through  2006  indicated  that  the  DoD 
awarded  471  contract  actions  for  auditing  services  totaling  approximately  $129.5  million 
in  obligated  funds  (Appendix  B).  From  this  information,  we  judgmentally  selected  five 
contracts  for  which  we  perfonned  a  detailed  review  of  the  capabilities  and  practices  in 
overseeing  them.  The  following  table  identifies  the  contracts  and  the  period  of  contract 
oversight  we  reviewed. 


Oversight  Reviewed  on  Selected  Contracts 


Contract  Number 

Statement  of  Work 

Short 

Form 

Oversight 
Done  bv 

Period 

Reviewed 

GS-23F-8127H  / 
HDEC05-06-F-0002 

Audit  of  the  Defense  Commissary  Agency 
Financial  Statements 

DeCA 

FS  Audit 

DeCA 

Jan  06  - 
Dec  07 

MDA230-02-A-000 1 

Audit  of  the  Defense  Finance  and 

Accounting  Service  Working  Capital  Fund 
Financial  Statements 

DFAS 

FS  Audit 

DFAS 

Mar  06  - 
Nov  07 

GS-23F-8132H/ 

DO72089 

Audit  of  the  Department  of  Defense 

Medicare  Eligible  Retiree  Flealth  Care  Fund 
Financial  Statements 

MERHCF 
FS  Audit 

DoD  OIG 

Mar  06  - 
Dec  06 

GS-23F-8132H/ 

D035793 

Information  Assurance  and  Compliance 

Audit  of  the  Defense  Departmental 

Reporting  System 

DDRS 

IA  Audit 

DoD  OIG 

Sep  04  - 
Oct  05 

N00421-05-D-0027 

Information  Assurance  and  Compliance 

Audit  of  the  Defense  Information  Systems 
Agency  Center  for  Computing  Services 

DISA 

IA  Audit 

DoD  OIG 

Dec  05  - 
Dec  06 

We  met  with  contracting  officer  representatives  and  other  officials  who  assisted  them  to 
discuss  the  procedures  they  used  to  oversee  contractors.  We  also  reviewed  oversight 
documentation.  We  evaluated  whether  the  oversight  staff: 


•  possessed  the  appropriate  technical  knowledge,  skills,  experience,  and  training 
necessary  to  be  competent  in  overseeing  the  contracted  audit  services. 

•  monitored  the  independence  of  key  staff  on  the  contractor's  audit  engagement 
team  as  well  as  the  organizational  independence  of  the  firm. 

•  monitored  the  qualifications  and  experience  of  key  staff  on  the  contractor's  audit 
engagement  team. 
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•  monitored  the  adequacy  of  the  contractor’s  quality  control  system. 

•  planned,  performed,  and  documented  their  oversight  (1)  commensurate  with  the 
contractor’s  objectivity  and  independence,  qualifications,  perfonnance  history, 
and  level  of  quality  control;  the  complexity  and  scope  of  the  contract 
requirements;  and  the  contract  performance  schedule;  and  (2)  to  assure  that  the 
contractor  complied  with  applicable  auditing  standards  and  the  contract. 

We  performed  this  review  from  December  2006  through  January  2008. 

Use  of  Computer-Processed  Data.  We  used  the  Federal  Procurement  Data  System- 
Next  Generation  (FPDS-NG)  as  a  source  for  data  on  DoD  contract  actions  for  auditing 
services.  Since  its  inception  in  1978,  FPDS  has  served  as  the  Government-wide  system 
for  collecting  Federal  procurement  data,  and  almost  since  its  beginning,  the  Government 
Accountability  Office  has  reported  concerns  about  it. 

The  Government  Accountability  Office  expressed  concerns  in  September  2005  about  the 
accuracy  of  FPDS-NG  data.  However,  we  relied  on  the  data  to  identify  contracting 
actions  because  we  believed  that  the  FPDS-NG  was  the  best  available  source  for  that 
information. 

Prior  Coverage 

Air  Force  Audit  Agency  Report  No.  F2006-0003-FD2000,  “Public  Accountant 
Contract  Audits,”  May  30,  2006.  The  audit  included  tests  of  internal  controls  over 
quality  assurance  monitoring,  the  Public  Accountant  Contract  Audit  selection  process, 
and  invoice  payment  accuracy  and  certification.  The  audit  did  not  identify  any  material 
departures  from  audit  standards  or  significant  administration  discrepancies;  therefore,  the 
report  did  not  contain  recommendations  for  corrective  action. 
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Appendix  B.  Analysis  of  DoD  Contracting  Activity 

for  FYs  2004  through  2006 


Analysis  of  DoD  contracting  activity  for  FYs  2004  through  2006  indicated  that  the  DoD 
awarded  471  contract  actions  for  auditing  services  totaling  approximately  $129.5  million 
in  obligated  funds.  To  detennine  this,  we  compiled  and  analyzed  procurement  data,  as 
well  as  identified  audit  services  contracts  for  which  the  DoD  OIG  is  overseeing.  From 
this  information,  we  judgmentally  selected  five  contracts  for  which  we  performed  a 
detailed  review  of  the  capabilities  and  practices  in  overseeing  them.  This  appendix 
summarizes  our  methods  of  analysis  and  selection. 

Procurement  Data.  We  compiled  data  from  the  Federal  Procurement  Data  System- 
Next  Generation  (FPDS-NG).  This  data  consisted  of  contract  actions  reported  by  DoD 
and  non-DoD  organizations. 17,18  The  following  table  shows  that  451  contract  actions 
with  total  obligated  funds  of  approximately  $109.3  million  comprised  the  data 
population. 


Table  B-l.  Combined  FYs  2004-06  Data  on  DoD  Contract  Actions  Coded  as  Auditing  Services 

Amount 

Funding  Component1 

Number  of 

Obligated 

Actions3 

(millions) 

Department  of  the  Army 

79 

$  9.1 

Department  of  the  Navy 

87 

14.8 

Department  of  the  Air  Force 

101 

14.4 

U.S.  Army  Corps  of  Engineers-Civil  Program 

13 

0.0 

Defense  Advanced  Research  Project  Agency 

9 

10.6 

Defense  Commissary  Agency 

1 

1.0 

Defense  Contract  Audit  Agency 

3 

0.1 

Defense  Contract  Management  Agency 

1 

0.2 

Defense  Finance  and  Accounting  Service 

46 

17.9 

Defense  Information  Systems  Agency 

5 

1.7 

Defense  Logistics  Agency 

23 

12.5 

Defense  Threat  Reduction  Agency 

2 

0.6 

Missile  Defense  Agency 

5 

1.9 

National  Geospatial-Intelligence  Agency 

3 

1.6 

Other2 

73 

22.9 

451 

$109.3 

1  If  data  element  left  blank.  Funding  Agency  Code  initialized  to  Contracting  Agency  Code. 

2  Other  DoD  Components  such  as  the  DoD  OIG,  Tricare  Management  Activity,  etc. 

3  Contract  actions  (awards,  orders,  modifications,  etc.)  connected  to  209  contracts. 

17  The  DoD  organizations  are  the  Departments  of  the  Army,  Navy,  and  Air  Force;  Defense  Logistics  Agency;  and 
Other  DoD  Agencies. 

18  The  non-DoD  organizations  are  the  Department  of  the  Interior,  Department  of  the  Treasury,  General  Services 
Administration,  and  National  Aeronautics  and  Space  Administration.  These  organizations  have  memorandums  of 
agreement  with  DoD  relating  to  interagency  acquisition  and  contracting  support. 
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We  analyzed  the  data  to  identify  contract  actions  coded  “R704”  for  “Auditing  Services” 
as  defined  by  the  FPDS-NG  Product  and  Service  Codes  Manual  and  the  DoD 
Components  who  funded  the  actions. 

DoD  OIG  Data  on  Audit  Services  Contracts.  We  also  compiled  data  from  the 
Program  Directors  within  the  Department  of  Defense  Office  of  the  Deputy  Inspector 
General  for  Auditing,  Defense  Business  Operations.  The  Directors  identified  another 
seven  contracts  for  auditing  services  totaling  20  actions  and  approximately  $20.2  million 
obligated  for  FYs  2004  through  2006.  These  items  are  not  included  in  the  data  shown  in 
Table  B-l  because  they  were  not  properly  coded  as  “R704”  for  “Auditing  Services”  in 
FPDS-NG.  Instead,  they  were  coded  as  other  types  of  services  such  as  accounting, 
professional,  and  research.  Therefore,  our  initial  data  analysis  did  not  identify  them.  The 
following  table  identifies  the  procurement  items  and  the  service  codes  applied  to  them. 


Table  B-2.  Combined  FYs  2004-06  Data  on  Audit  Services  Contracts 
Coded  as  Other  Types  of  Services  in  FPDS-NG 


FPDS-NG 

Number  of 

Amount 

Obligated 

Contract  Number 

Code 

Actions 

(millions) 

GS-23F-0165N  /  D035438 

R703 

8 

$  5.6 

GS-23F-0165N  /D036008 

R703 

4 

0.7 

HHM402-04-A-0033  /  0001 

AE21,  D399 

3 

2.2 

HHM402-04-A-0033  /  0002 

R499 

1 

0.4 

N00421-05-D-0020  /  0001-02 

R499 

2 

1.0 

N00421-05-D-0025  /  0001 

R499 

1 

6.5 

N00421-05-D-0027  /  0001 

R499 

1 

3.8 

20 

$20.2 

AE21  Research  and  Development/Product  or  Service  Improvement 
D399  Other  Automatic  Data  Processing  and  Telecommunications  Services 
R499  Other  Professional  Services 
R703  Accounting  Services 


Selection  Process.  From  the  procurement  data  coded  as  auditing  services,  we  selected 
unique  procurement  instrument  ID  numbers  from  the  data  population  for  which  related 
actions  collectively  amounted  to  $1  million  or  more  in  obligated  funds.  This  selection 
consisted  of  29  unique  instruments  totaling  121  actions  and  $81.1  million  obligated,  or 
74.1  percent  of  the  approximately  $109.3  million  obligated  for  FYs  2004  through  2006. 
For  each  instrument,  we  determined  whether  the  contracted  services  represented  an  audit 
or  attestation  engagement  under  Government  Auditing  Standards  by  reviewing  contract 
schedules  of  services  and  statements  of  work.  We  also  researched  the  contractors’ 
Internet  sites  to  determine  whether  they  marketed  auditing  services.  Of  the  29,  nine 
inferred  contracted  audit  services  from  which  we  judgmentally  selected  four  for  detailed 
review  of  related  oversight.  Overall,  we  discounted  those  instruments  that  closed  out 
during  our  review  period,  as  well  as  those  inferring  nonaudit  work  such  as  accounting, 
financial  management,  or  technical  analytic  support  services.  From  the  additional  seven 
audit  services  contracts  identified  by  the  DoD  OIG,  we  judgmentally  selected  one  for 
detailed  review  of  related  oversight.  We  focused  on  contracts  for  which  related  actions 
collectively  amounted  to  $  1  million  or  more  in  obligated  funds  and  discounted  those  with 
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work  still  in  progress  and  work  completed  before  FY  2007.  Appendix  A  identifies  the 
contracts  we  selected  for  review. 
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